The detection of duplicates in document image databases
Identifieur interne : 002085 ( Main/Exploration ); précédent : 002084; suivant : 002086The detection of duplicates in document image databases
Auteurs : David Doermann [États-Unis] ; Huiping Li [États-Unis] ; Omid Kia [États-Unis]Source :
- Image and Vision Computing [ 0262-8856 ] ; 1997.
Descripteurs français
- Pascal (Inist)
- Wicri :
- topic : Base de données.
English descriptors
- KwdEn :
Abstract
Document imaging technology has developed to the point where it is not uncommon for organizations to scan large numbers of documents into databases with little or no index information. This may be done for archival purposes with an index as simple as a case number, or with the ultimate goal of automatically extracting index information for content-based queries. Maintaining the integrity of such a database is difficult, however, especially in a distributed environment where copies of the same documents may be scanned at different times. In this paper we present a novel approach to detecting duplicate documents in very large databases using only features extracted from the image. The method is based on a robust `signature' extracted from each document image which is used to index into a table of previously processed documents. The system is able to deal with differences between scanned document instances such resolution, skew and image quality. The approach has a number of advantages over OCR or other recognition-based methods including speed and robustness to imaging distortions. To justify the approach and demonstrate its scalability, we have developed a simulator which allows us to change parameters of the system and examine performance while processing millions of document signatures. A complete system has been implemented and tested on a collection of technical articles and memos.
Url:
DOI: 10.1016/S0262-8856(98)00054-7
Affiliations:
Links toward previous steps (curation, corpus...)
- to stream Istex, to step Corpus: 000186
- to stream Istex, to step Curation: 000183
- to stream Istex, to step Checkpoint: 001589
- to stream Main, to step Merge: 002202
- to stream PascalFrancis, to step Corpus: 000848
- to stream PascalFrancis, to step Curation: 000B48
- to stream PascalFrancis, to step Checkpoint: 000808
- to stream Main, to step Merge: 002402
- to stream Main, to step Curation: 002085
Le document en format XML
<record><TEI wicri:istexFullTextTei="biblStruct"><teiHeader><fileDesc><titleStmt><title>The detection of duplicates in document image databases</title>
<author><name sortKey="Doermann, David" sort="Doermann, David" uniqKey="Doermann D" first="David" last="Doermann">David Doermann</name>
<affiliation><country>États-Unis</country>
<placeName><settlement type="city">College Park (Maryland)</settlement>
<region type="state">Maryland</region>
</placeName>
<orgName type="university" n="3">Université du Maryland</orgName>
</affiliation>
</author>
<author><name sortKey="Li, Huiping" sort="Li, Huiping" uniqKey="Li H" first="Huiping" last="Li">Huiping Li</name>
</author>
<author><name sortKey="Kia, Omid" sort="Kia, Omid" uniqKey="Kia O" first="Omid" last="Kia">Omid Kia</name>
</author>
</titleStmt>
<publicationStmt><idno type="wicri:source">ISTEX</idno>
<idno type="RBID">ISTEX:14D436D3D8870783B3BB81632855ECBEC0DF7FAD</idno>
<date when="1998" year="1998">1998</date>
<idno type="doi">10.1016/S0262-8856(98)00054-7</idno>
<idno type="url">https://api.istex.fr/document/14D436D3D8870783B3BB81632855ECBEC0DF7FAD/fulltext/pdf</idno>
<idno type="wicri:Area/Istex/Corpus">000186</idno>
<idno type="wicri:Area/Istex/Curation">000183</idno>
<idno type="wicri:Area/Istex/Checkpoint">001589</idno>
<idno type="wicri:doubleKey">0262-8856:1998:Doermann D:the:detection:of</idno>
<idno type="wicri:Area/Main/Merge">002202</idno>
<idno type="wicri:source">INIST</idno>
<idno type="RBID">Pascal:98-0532763</idno>
<idno type="wicri:Area/PascalFrancis/Corpus">000848</idno>
<idno type="wicri:Area/PascalFrancis/Curation">000B48</idno>
<idno type="wicri:Area/PascalFrancis/Checkpoint">000808</idno>
<idno type="wicri:doubleKey">0262-8856:1998:Doermann D:the:detection:of</idno>
<idno type="wicri:Area/Main/Merge">002402</idno>
<idno type="wicri:Area/Main/Curation">002085</idno>
<idno type="wicri:Area/Main/Exploration">002085</idno>
</publicationStmt>
<sourceDesc><biblStruct><analytic><title level="a">The detection of duplicates in document image databases</title>
<author><name sortKey="Doermann, David" sort="Doermann, David" uniqKey="Doermann D" first="David" last="Doermann">David Doermann</name>
<affiliation wicri:level="4"><country xml:lang="fr">États-Unis</country>
<wicri:regionArea>Language and Media Processing Laboratory, Center for Automation Research, University of Maryland, College Park, MD 20742-3275</wicri:regionArea>
<placeName><region type="state">Maryland</region>
<settlement type="city">College Park (Maryland)</settlement>
</placeName>
<orgName type="university">Université du Maryland</orgName>
<placeName><settlement type="city">College Park (Maryland)</settlement>
<region type="state">Maryland</region>
</placeName>
<orgName type="university" n="3">Université du Maryland</orgName>
</affiliation>
</author>
<author><name sortKey="Li, Huiping" sort="Li, Huiping" uniqKey="Li H" first="Huiping" last="Li">Huiping Li</name>
<affiliation wicri:level="4"><country xml:lang="fr">États-Unis</country>
<wicri:regionArea>Language and Media Processing Laboratory, Center for Automation Research, University of Maryland, College Park, MD 20742-3275</wicri:regionArea>
<placeName><region type="state">Maryland</region>
<settlement type="city">College Park (Maryland)</settlement>
</placeName>
<orgName type="university">Université du Maryland</orgName>
</affiliation>
</author>
<author><name sortKey="Kia, Omid" sort="Kia, Omid" uniqKey="Kia O" first="Omid" last="Kia">Omid Kia</name>
<affiliation wicri:level="4"><country xml:lang="fr">États-Unis</country>
<wicri:regionArea>Language and Media Processing Laboratory, Center for Automation Research, University of Maryland, College Park, MD 20742-3275</wicri:regionArea>
<placeName><region type="state">Maryland</region>
<settlement type="city">College Park (Maryland)</settlement>
</placeName>
<orgName type="university">Université du Maryland</orgName>
</affiliation>
</author>
</analytic>
<monogr></monogr>
<series><title level="j">Image and Vision Computing</title>
<title level="j" type="abbrev">IMAVIS</title>
<idno type="ISSN">0262-8856</idno>
<imprint><publisher>ELSEVIER</publisher>
<date type="published" when="1997">1997</date>
<biblScope unit="volume">16</biblScope>
<biblScope unit="issue">12–13</biblScope>
<biblScope unit="page" from="907">907</biblScope>
<biblScope unit="page" to="920">920</biblScope>
</imprint>
<idno type="ISSN">0262-8856</idno>
</series>
<idno type="istex">14D436D3D8870783B3BB81632855ECBEC0DF7FAD</idno>
<idno type="DOI">10.1016/S0262-8856(98)00054-7</idno>
<idno type="PII">S0262-8856(98)00054-7</idno>
</biblStruct>
</sourceDesc>
<seriesStmt><idno type="ISSN">0262-8856</idno>
</seriesStmt>
</fileDesc>
<profileDesc><textClass><keywords scheme="KwdEn" xml:lang="en"><term>Algorithm</term>
<term>Algorithm performance</term>
<term>Computer simulation</term>
<term>Database</term>
<term>Database query</term>
<term>Experimental study</term>
<term>Image processing</term>
<term>Indexing</term>
<term>Pairing</term>
<term>Pattern extraction</term>
<term>Pattern recognition</term>
</keywords>
<keywords scheme="Pascal" xml:lang="fr"><term>Algorithme</term>
<term>Appariement</term>
<term>Base donnée</term>
<term>Etude expérimentale</term>
<term>Extraction forme</term>
<term>Indexation</term>
<term>Interrogation base donnée</term>
<term>Performance algorithme</term>
<term>Reconnaissance forme</term>
<term>Simulation ordinateur</term>
<term>Traitement image</term>
</keywords>
<keywords scheme="Wicri" type="topic" xml:lang="fr"><term>Base de données</term>
</keywords>
</textClass>
<langUsage><language ident="en">en</language>
</langUsage>
</profileDesc>
</teiHeader>
<front><div type="abstract" xml:lang="en">Document imaging technology has developed to the point where it is not uncommon for organizations to scan large numbers of documents into databases with little or no index information. This may be done for archival purposes with an index as simple as a case number, or with the ultimate goal of automatically extracting index information for content-based queries. Maintaining the integrity of such a database is difficult, however, especially in a distributed environment where copies of the same documents may be scanned at different times. In this paper we present a novel approach to detecting duplicate documents in very large databases using only features extracted from the image. The method is based on a robust `signature' extracted from each document image which is used to index into a table of previously processed documents. The system is able to deal with differences between scanned document instances such resolution, skew and image quality. The approach has a number of advantages over OCR or other recognition-based methods including speed and robustness to imaging distortions. To justify the approach and demonstrate its scalability, we have developed a simulator which allows us to change parameters of the system and examine performance while processing millions of document signatures. A complete system has been implemented and tested on a collection of technical articles and memos.</div>
</front>
</TEI>
<affiliations><list><country><li>États-Unis</li>
</country>
<region><li>Maryland</li>
</region>
<settlement><li>College Park (Maryland)</li>
</settlement>
<orgName><li>Université du Maryland</li>
</orgName>
</list>
<tree><country name="États-Unis"><region name="Maryland"><name sortKey="Doermann, David" sort="Doermann, David" uniqKey="Doermann D" first="David" last="Doermann">David Doermann</name>
</region>
<name sortKey="Kia, Omid" sort="Kia, Omid" uniqKey="Kia O" first="Omid" last="Kia">Omid Kia</name>
<name sortKey="Li, Huiping" sort="Li, Huiping" uniqKey="Li H" first="Huiping" last="Li">Huiping Li</name>
</country>
</tree>
</affiliations>
</record>
Pour manipuler ce document sous Unix (Dilib)
EXPLOR_STEP=$WICRI_ROOT/Ticri/CIDE/explor/OcrV1/Data/Main/Exploration
HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 002085 | SxmlIndent | more
Ou
HfdSelect -h $EXPLOR_AREA/Data/Main/Exploration/biblio.hfd -nk 002085 | SxmlIndent | more
Pour mettre un lien sur cette page dans le réseau Wicri
{{Explor lien |wiki= Ticri/CIDE |area= OcrV1 |flux= Main |étape= Exploration |type= RBID |clé= ISTEX:14D436D3D8870783B3BB81632855ECBEC0DF7FAD |texte= The detection of duplicates in document image databases }}
This area was generated with Dilib version V0.6.32. |